\chapter{Theory}
\label{sec:theory}
This section describes the methods chosen for detecting people and gestures.

\section{People Detection}
Automatically detecting people in an image is an active research field in computer vision, with applications for example in surveillance. The problem includes tasks such as determining what parts of the image belong to the background and foreground, and detecting people that may be of different shapes and in different positions. The use of depth image data can simplify people detection.


\subsection{Blobs}
In image processing, blob detection is a method for finding connected components in an image. The library used in the implementation uses the algorithm by Chang et. al. \cite{Chang2004206} to find 8-connected blobs. This algorithm utilizes contour tracing to find components that can be labelled in one pass. The algorithm works in linear time.

\section{Gesture recognition}
Several different methods have been suggested for gesture recognition. Some of these methods are based on unsupervised learning algorithms. For this project, Canonical correlation analysis was chosen. 

\subsection{Canonical correlation analysis}
Canonical correlation analysis (CCA) is a method which can be used to find the maximum mutual information between two signals. The method was introduced by Hotelling in 1936, and has more recently been found useful in signal processing tasks, such as described by Borga \cite{borga98}.

The method works by finding basis vectors for two sets of variables, such that when the variables are projected onto these basis vectors their correlation is maximized. The difference between CCA and correlation analysis is that correlation analysis is coordinate system dependent, whereas the projection onto the new basis in CCA removes this dependency. CCA is invariant to affine transformations, which makes it attractive in an application where scaling and translations of gestures are common. 

Two random variables, x and y, have the total covariance matrix
\begin{equation}
\mathbf{C}= \begin{bmatrix} \mathbf{C}_{xx} &\mathbf{C}_{xy}\\\mathbf{C}_{yx} & \mathbf{C}_{yy}\end{bmatrix}= 
E \begin{bmatrix} \begin{pmatrix} \mathbf{x} \\\mathbf{y} \end{pmatrix} \begin{pmatrix}\mathbf{x} \\\mathbf{y}\end{pmatrix}^T\end{bmatrix}
\end{equation}
They are projected onto two vectors, $\mathbf{w}_x$ and $\mathbf{w}_y$.The function to be maximized is the correlation between the projections of $x$ and $y$, $\mathbf{x}^T\mathbf{w}_x$ and $\mathbf{y}^T\mathbf{w}_x$
\begin{align}
\nonumber \rho &= \frac{E\left[xy \right] }{\sqrt{E\left[x^2 \right]E\left[ y^2\right]  }} = 
\frac{E\left[ \mathbf{\hat{w}}_{x}^{T} \mathbf{xy}^T \hat{\mathbf{w}}_y\right]} 
{\sqrt{E\left[ \mathbf{\hat{w}}_{x}^{T} \mathbf{xx}^T \hat{\mathbf{w}}_x\right]
E\left[ \mathbf{\hat{w}}_{y}^{T} \mathbf{yy}^T \hat{\mathbf{w}}_y\right] }} \\
&=\frac{\mathbf{w}_x^T\mathbf{C}_{xy}\mathbf{w}_y}
{\sqrt{	\mathbf{w}_x^T\mathbf{C}_{xx}\mathbf{w}_x
			\mathbf{w}_y^T\mathbf{C}_{yy}\mathbf{w}_y}}
\end{align}
Setting the derivatives to zero gives the equation system

\begin{equation}
\label{eq:system}
\left\{
  \begin{array}{l}
    \mathbf{C}_{xy}\mathbf{\hat{w}_y} = \rho \lambda \mathbf{C}_{xx}\mathbf{\hat{w}_x}\\
    \mathbf{C}_{yx}\mathbf{\hat{w}_x} = \rho \lambda \mathbf{C}_{yy}\mathbf{\hat{w}_y}\\
  \end{array} \right.
\end{equation}
Equation \ref{eq:system} can be rewritten as

\begin{equation}
\label{eq:bleh}
\left\{
  \begin{array}{l}
    \mathbf{C}_{xx}^{-1}\mathbf{C}_{xy}\mathbf{C}_{yy}^{-1}\mathbf{C}_{yx}\mathbf{\hat{w}_x} = \rho^2 \mathbf{\hat{w}_x}\\
    \mathbf{C}_{yy}^{-1}\mathbf{C}_{yx}\mathbf{C}_{xx}^{-1}\mathbf{C}_{xy}\mathbf{\hat{w}_y} = \rho^2 \mathbf{\hat{w}_y}\\
  \end{array} \right.
\end{equation}
From Equation \ref{eq:bleh} $\mathbf{\hat{w}}_x$ and $\mathbf{\hat{w}}_y$ can be found as the eigenvectors of the matrices $\mathbf{C}_{xx}^{-1}\mathbf{C}_{xy}\mathbf{C}_{yy}^{-1}\mathbf{C}_{yx}\mathbf{\hat{w}_x}$ and $\mathbf{C}_{yy}^{-1}\mathbf{C}_{yx}\mathbf{C}_{xx}^{-1}\mathbf{C}_{xy}\mathbf{\hat{w}_y}$. The eigenvalues, $\rho^2$ that correspond to these eigenvectors are the squared maximum canonical correlations. 

A gesture is seen as a set of feature points, and to classify a gesture a captured set of points is compared to sets of previoulsy stored prototypes by finding the maximum correlation with each prototype. The group of prototypes where the maximum correlation is found is the gesture which is taking place, if it is above some threshold.




